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EXECUTIVE  SUMMARY  i 

Mamagement  of  embedded  computer  software  in  weapon  systems 
is  an  important  and  current  consideration.  A large  portion  | 

of  the  Defense  budget  is  spent  in  software  acquisition  and 
support  and  the  management  techniques  for  making  this  a 

productive  and  cost  effective  process  must  be  developed.  j 

» This  paper  addresses  the  issue  from  two  aspects.  First,  a j 

study  of  the  current  trends  in  software  management  was  con-  s 

• i 

ducted  with  particular  emphasis  on  DoD  sponsored  activities.  ' 

' Some  major  cornerstones  of  successful  software  management 

are  developed  throughout  the  paper  and  are  summarized  in  the  j 

I 

conclusion  section  of  this  paper.  Secondly,  the  software 
management  and  development  approach  being  taken  in  the  Navy's 
AEGIS  Combat  System  was  studied.  This  proved  to  be  an 

informative  and  constructive  endeavor  that  tends  to  give  j 

real-world  meaning  to  the  application  of  software  management  j 

techniques.  ! 
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SECTION  I 


INTRODUCTION 


1.  Purt'ose 

The  purpose  of  this  paper  is  two-fold.  First  sm  attempt 
is  made  to  examine  the  issue  of  software  management  from  a 
"textbook"  viewpoint.  Recent  publications,  articles,  reports, 
books  and  studies  will  be  reviewed  and  anaylzed  for  deter- 
mining current  trends  in  the  area  of  software  management. 

Under  the  full  realization  that  no  cookbook  procedure  will 
be  found,  hope  remains  that  some  basic  cornerstones  will  be 
identified  that  are  essential  to  effective  management  of 
software.  An  attempt  to  arrive  at  a singular  solution  to  the 
problem  would  be  an  act  of  simplism.  However,  if  an  inter- 
related group  of  events  can  be  established  that  identify  the 
hurdles  that  must  be  cleared,  progress  will  have  been  made. 

Secondly,  a real-world  application  of  software  m^Aagement 
will  be  discussed.  One  of  the  current  major  weapon  system 
acquisitions  in  the  Navy,  the  AEGIS  Combat  System,  will 
provide  the  data  base  for  this  discussion.  The  intent  of 
this  portion  of  the  paper  is  not  to  compare  software  manage- 
ment in  AEGIS  with  the  check-off  list  of  good  practices 
because  by  previous  admission,  no  such  list  exist.  It  is 
intended  to  be  an  education  process  for  the  author  suad  the 
reader.  Perhaps  some  of  the  "textbook"  trends  will  be 
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reinforced  by  the  AEGIS  approach > perhaps  some  new  lights 
will  glow. 

As  for  the  motivation  to  choose  a topic  such  as  software 
management,  tw^o  statements  of  need  are  offered! 

J.  S.  Gansler  OASD(I  & L) i 

The  functional  applications  of  software  within 
Department  of  Denfense  (DoD)  weapon  systems  pervade 
almost  every  program. 

The  most  critical  issue  facing  DoD  is  the  increasing 
use  of  and  dependency  on  software  in  weapon  systems 
v;ithout  the  proven  management  and  production  methods 
necessary  to  control  its  direct  and  indirect  cost. 

The  second  major  issue  concerns  the  need  to  convert 
software  from  an  art  to  a technology.  The  notion 
that  software  advancements  are  the  sole  domain  of 
"rugged  individualists"  or  "artists"  must  be 
discarded.  (6il)l 

DODD  5000.29 

Annual  expenditures  by  DoD  on  the  design,  develop- 
ment, acquisition,  management,  and  operational 
support  of  computer  resources  embedded  within,  and 
integral  to  weapons,  communications,  command  and 
control,  and  intelligence  sensor  systems  are  measured 
in  the  billons  of  dollars.  Unreliabilitv,  particularly 
of  software,  diminishes  DoD  mission  effectiveness  in 
many  major  Defense  Systems. 

Computer  resources  in  Defense  Systems  must  be  managed 
as  elements  or  subsystems  of  major  importance  during 
conceptual,  validation,  full-scale  developm.ent, 
production,  deployment,  and  support  phases  of  the 
life  cycle,  with  particular  emphasis  on  computer 
software  and  its  integration  with  the  surrounding 
hardware.  (8i2) 


1*  This  notation  will  be  used  throughout  this  paper  for 
sources  of  quotations  and  major  references.  The  first  number 
is  the  source  listed  in  the  bibliography.  The  second  number, 
if  listed,  is  the  page  in  the  reference. 
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Goals 


The  author  of  this  paper  has  five  years  experience  in 
providing  the  life  support  for  software  contained  in  Navy 
missile  systems.  This  has  been  primarily  in  the  acquisition 
phases  of  full  scale  developments  production  and  deployment. 
This  paper  is  viewed  as  an  educational  opportunity  to  examin- 
ing the  entire  acquisition  process  of  major  weapon  systems, 
including  conception  and  validation,  and  the  considerations 
given  to  software  management  therein.  It  was  felt  that  an 
examination  of  the  current  literature  was  a prerequisite  for 
such  a venture  and  forms  the  basis  for  the  first  of  two  goals, 
ie,  examine  the  current  trends  in  software  management. 

Once  the  textbook  teachings  are  filed  away  it  becomes  a 
natural  follow  on  to  examine  the  real  world.  Hence  goal 
number  twoj  understand  the  software  management  approach  in  a 
real  world  application.  The  Navy  AEGIS  Combat  System  was 
chosen  for  several  reasons  which  include  its  currentness,  its 
complexity  and  its  heavy  use  of  computer  programs. 

3.  Scope 

This  paper  is  intended  to  provide  a condensation  of  the 
current  trends  in  software  management  and  to  relate  the  find- 
ings of  an  educational  examination  of  a real  world  application. 
It  is  intended  for  the  use  of  anyone  involved  in  software 
management  of  embedded  computer  programs.  A point  made  in  the 
Purpose  section  needs  to  be  re-emphasized.  Is  is  not  in  the 
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scope  of  this  paper  to  critique  the  software  management 
approach  being  taken  in  the  AEGIS  Combat  System.  The  review 
of  a real  world  application  is  being  used  to  provide  additional 
coverage  of  the  topic. 

ij-.  Limitations 

Due  to  the  author’s  occupational  interests  and  the  limited 
time  available  for  the  generation  of  this  paper  the  subject 
matter  has  been  limited  to  embedded  software  applications. 

The  interrelation  between  the  weapon  system  acquisition  cycle 
and  the  software  acquisition  cycle  will  necessitate  that  a 
certain  amount  of  the  big  picture  to  be  portrayed.  We  are 
concerned  here  with  the  software  management  of  the  embedded 
computer  programs  in  a weapon  system  acquisition. 

The  choice  of  a Navy  application  for  the  real  world  part 
of  the  paper  has  been  made  for  the  reasons  previously  stated. 
This  is  not  to  imply  that  the  concepts  and  realizations 
presented  in  this  paper  do  not  apply  equally  as  well  to  the 
other  Services. 

5.  Organization  and  Methodology 

In  addition  to  this  introductory  section,  the  paper  has 
been  organized  into  two  major  sections  and  a summary  section. 
Section  II  examines  the  current  trends  in  the  area  of  software 
management  from  a textbook  approach.  Various  types  of 
reference  literature,  including  articles,  reports,  study 
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papers,  textbooks  and  DoD  publication  and  directives,  were 
studied  and  analyized  in  an  attempt  to  gain  insight  on  the 
key  considerations  in  software  management.  Many  areas  of 
software  management  and  development  will  be  discussed  with 
varying  degrees  of  intensity.  Of  particular  interest  will 
be  the  identification  of  basic  cornerstones  of  successful 
software  management  and  their  relation  to  the  Defense  acquisi- 
tion cycle.  Any  conclusions  drawn  will  be  based  on  the 
authors  evaluation  of  the  information  studied  combined  with 
personal  experience  and  involvement  in  the  software  acquisi- 
tion and  management  arenas. 

Section  III  will  be  an  educational  process  in  which  a 
real  world  weapon  system  acquisition  will  be  studied.  In 
order  that  such  an  effort  be  accomplished,  it  was  necessary 
to  obtain  and  study  the  available  documentation  for  the 
selected  acquisition  process.  In  addition,  personal  inter- 
views were  conducted  with  the  Program  Manager's  office  and 
the  Navy  Lead  Laboratory.  The  interview  questionnaire  used 
for  this  purpose  is  enclosed  as  Appendix  A to  this  report. 

Section  IV  is  the  summary  containing  conclusions  and 


recommendations. 


SECTION  II 

CURRENT  TRENDS  IN  SOFTWARE  MANAGEMENT 
1.  Background 

The  application  of  computing  technology  to  our  evolving 
life  style  has  been  natural  and  rapid.  Man  has  continually 
been  in  search  of  ways  to  advance  his  state  of  being  and 
improve  his  environment.  Computational  advancements  have 
been  based  primarly  on  the  physical  sciences  which  have  had 
the  effect  of  creating  faster  and  larger  computing  systems 
with  which  to  solve  our  problems.  This  has  been  a successful 
and  efficient  evolution,  for  the  most  part,  because  of  the 
well  developed  skills  that  were  directed  to  the  developments, 
Kan  has  been  actively  concerned  and  involved  in  advancing  the 
state  of  the  art,  particularly  in  areas  that  are  not  art. 
Generation  of  smaller,  faster  and  more  reliable  circuits  are 
a direct  result  of  our  time  tested  techniques  in  hardware 

; ' 

/ design,  development  and  testing.  We  know  how  to  specify  what 

r 

we  want,  how  to  build  it,  how  to  nurture  our  technology  base 
for  future  breakthroughs,  how  to  test  it  and  how  to  maintain 
it.  In  summary  we  are  very  effective  in  advancing  the  hard- 
ware aspect  of  computing  technology. 

In  the  world  of  weapon  system  acquisition  it  is  not  too 
difficult  for  officals  to  visualize  and  understand  the  process 
of  hardware  development  and  hence,  the  application  of  manage- 
ment skills,  which  have  been  developed  from  our  previous 
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experiences  and  experiments  involving  hardware  technology, 
often  prove  to  be  successful.  We  are  adept  at  specifing  a 
widget  to  be  twelve  meters,  plus  or  minus  two  centimeters. 

It  is  fairly  simple  to  determine  if  the  manufacturing 
technology  (state  of  the  art)  can  make  it.  Also  it  is  easy 
to  test  I we  just  measure  it  against  a standard  unit.  We 
certainly  know  how  to  maintain  it  by  establishing  a physical 
set  of  conditions  to  which  it  must  always  conform.  If  it 
doesn’t,  we  tweek  it,  repair  it  or  replace  it.  The  manage- 
ment approach  throughout  this  entire  process  is  based  on  the 
physical  presence  of  the  acquisition.  If  we  need  to  know  how 
our  widget  is  coming  along,  we  take  a look  at  it. 

All  the  above  is  not  intended  to  simplify  the  management 
of  hardware  acquisitions.  One  of  the  things  we  know  to  be  fact 
is  change  and  with  it  we  must  adapt.  This  fact  makes  manage- 
ment of  anything  a real  challenge.  The  point  of  the  above  is 
to  provide  a jumping  off  point  for  the  topic  of  this  paper,  ie, 
software  management.  If  we  accept  the  position  that  in  fact 
we  are  experiencing  undue  difficulties  in  the  area  of  software 
management,  we  can  direct  our  attention  to  possible  causes 
and  corrections.  Most  any  of  the  references  in  the  bibliography 
go  a long  way  toward  the  affirmation  of  this  position.  We  will 
take  advantage  of  some  of  the  k».'v  points  made  in  the  literature 
as  we  further  develope  the  subject. 


2 . Guidance  Documents 

The  following  statement  of  concern  contained  in  an 

memoranJuTi  from  the  Office  of  the  Secretary  of  Defense  in 

1974  characterizes  the  increased  emphasis  on  the  part  of  DoD 

on  the  solution  to  the  problems  of  increasing  software  cost, 

schedule  slippages  and  technical  reliability  that  many  weapon 

system  acquisitions  are  encountering. 

The  sharply  rising  costs  of  software  programs  in 
the  weapon  system  acquisition  process,  with  respect 
to  acquisition  procedures,  development  and  mainten- 
ance of  such  software,  and  the  increasing  importance 
of  the  software  role  in  the  overall  mission  effect- 
iveness of  Department  of  Defense  (DoD)  weapon  systems 
constitutes  serious  technical  and  management  problems 
that  must  be  solved  if  we  are  to  have  the  weapon 
systems  that  are  needed  for  national  security  (lOil) 

Many  of  the  management  and  technical  problems  associated 

with  the  acquisition  of  weapon  system  software  arise  from  the 

attempt  to  use  hardware  acquisition  techniques  and  guidance 

documents.  The  Department  of  Defense  and  Service  Components 

have  many  policies  and  procedures  and  related  documentation 

(Directives,  Instructions,  Standards,  etc.)  that  are  concerned 

with  hardware  acquisition.  Very  few  of  these  deal  with 

computer  software  acquisition  and  even  fewer  deal  with 

embedded  computer  software.  In  recent  years  there  has  been 

a trend  to  treat  Automatic  Data  Processing  (ADP)  separately 

from  Embedded  Computer  System  (ECS),  This  is  a step  in  the 

right  direction  in  that  a basic  difference  exists  in  the  two 

areas.  In  ADP  the  product  is  a computing  system  and  the 
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associated  programming.  In  ECS  the  software  product  is  often 
difficult  to  define,  or  evaluate  for  that  matter,  except  in 
terms  of  the  weapon  system  in  which  it  resides. 

The  OSD  memorandum  referenced  above  established  a joint 
DoD/Service  Software  Steering  Committee  with  an  objective  to 
determine  methods  to  reduce  and  control  software  costs  and 
improve  software  reliability,  standardization  and  maintain- 
ability. The  initial  task  of  the  committee  was  to  oversee 
software  management  studies  by  the  Applied  Physics  Laboratory 
of  John  Hopkins  University  (APL/JHU)  and  the  Kitre  Corporation. 
Each  was  to  conduct  separate,  but  coordinated,  four  month 
studies  to  identify  and  define i 

1.  the  nature  of  the  critical  software  problems 
facing  the  DoD, 

2.  the  principal  factors  contributing  to  the  problems, 

3.  the  high  payoff  areas  and  alternatives  available, 

4.  and  the  management  instruments  and  policies  that 
are  needed  to  define  and  bound  the  functions, 
responsibilities  and  mission  areas  of  syste-"  soft- 
ware management  ( 10 i 1 ) 

The  second  phase  of  the  study  was  an  indepth  study  into 
the  critical  areas  identified  as  a result  of  the  initial  four 
month  studies, 

APL/jHU  conducted  their  study  in  three  parts.  First,  ten 
recent  DoD  sponsored  studies  in  software  management  were 
analyzed.  Secondly,  the  software  design  and  management  approach 
being  employed  in  ten  Navy  and  two  Army  Weapon  System.s  were 
reviewed.  Thirdly,  discussions  with  service  and  industry 
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organizations  involved  in  software  acquisition  were  conducted. 
The  findings  presented  in  the  study  report  included  seven 
categories  of  problem  areas  with  recommended  actions  in  each. 
In  essence  the  categories  and  recommendations  are  as  follows 
(3«2-l)i 

a.  Management  Policy  - Comprehensive  analysis  and 
definition  of  software  requirements  down  to  the 
major  function  level  should  be  conducted  in  the 
Validation  Phase.  Promote  software  visibility 
in  terms  of  configuration  control  items,  DSARC 
reviews,  design  reviews  and  other  aspects  of 
acquisition  management.  Specify  that  software 
be  designated  as  Configuration  Items  (Cl)  and 
deliverables  during  Full  Scale  Development  Phase 
including  computer  programs  and  computer  data 
for  operational  software  development,  support 
software  and  test  and  integration  software. 

b.  Acquisition  Planning  - Establish  milestones  and 
achievement  criteria  in  the  Pull  Scale  Develop- 
ment Phase  to  ensure  the  proper  sequence  of 
analysis,  design,  implementation,  test  and 
integration.  Require  a detailed  Computer  System 
Resource  Development  Plan  as  part  of  the  bid 
package  of  Full  Scale  Development  contracts. 

c.  Systems  Engineering  - Establish  functional  segments 
in  accordance  with  the  operational  requirements 
and  conduct  hardware/software  trade  off  analysis 
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in  the  Validation  Phase,  Ensure  the  software 
design  makes  provision  for  growth  to  accommodate 
uncertainties  and  changes  in  system  re-iuirernents. 
Specify  the  use  of  modular  software  architecture 
and  top-down  design. 

d.  Implementation  Procedures  - Include  in  Full  Scale 
Development,  provision  for  adequate  modern  support 
tools  and  facilities  and  the  transferring  of  same 
to  the  Operational  Support  (Maintenance)  Agent. 
Apply  highly  disciplined  engineering  practices  to 
software  development  and  require  a progressive 
system  integration  and  test  capability. 

e.  Program  Management  Support  - Require  appropriate 
technical  staffing  in  the  Program  Manager's 
Office  with  experience  in  systems  engineering 
and  software  development.  This  should  include 

a Systems  Engineering  Agent  and  a Software 
Operational  Support  Agent. 

f.  Acquisition  Management  Standards  - Establish  a 
set  of  requirements  and  criteria  to  be  applied 
in  the  acquisition  and  support  of  weapon  system 
computer  resources  by  all  services  and  prepare 
a series  of  handbooks  and  guides  covering 
important  aspects  of  software  acquisition. 

g.  Development  of  Tools  and  Techniques  - Support 
the  development  of  software  test  and  validation 
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1 
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J 

tools  to  reduce  the  cost  and  time  involved  in 
software  verification. 

The  Mitre  study  resulted  in  the  identifica- 
tion of  four  high  payoff  areas  in  the  software 
acquisition  process  in  which  corrective  actions 
should  be  taken  (13«1) 

a.  Software  Performance  Specification  - 
Establishment  and  consistent  application  of 

engineering  principals/practices  to  the  , 

1 

process  of  specifying  and  validation  soft- 
ware requirements. 

b.  Software  Acquisition  Planning  - Early  and 
complete  software  life  cycle  planning  and 
establishment  and  application  of  management 
practices/strategies  specifically  for  soft- 
ware, 

c.  Software  Technology  - High  leverage  technology 
programs  needed  to  further  improve  software 
practices  and  techniques. 

d.  Personnel  - Provision  for  knowledgeable  aind 
experienced  DoD  software  management  and 
software  engineering  personnel. 

In  addition  to  the  two  reports  described  above  there  have 
been  many  similiar  studies  undertaken  at  all  levels  in  the 
Department  of  Defense,  Joint  Logistics  Commanders,  Service 
Components  and  Industry.  Department  of  Defense  Directive 
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5000,29  of  April  197^  is  the  result  of  the  DoD  Software 
steering  Committee  actions  and  recommendations.  Some  of  the 
major  areas  addressed  are  (8)j 

* Validation  of  computer  resource  requirements, 
including  software,  interface  control  and 
integration  methodology  definition  will  be 
conducted  during  the  Concert  and  Validation 
Phases,  prior  to  DSARC  II 

* Computer  software  will  be  specified  and  treated 
as  configuration  items. 

* A computer  resource  plan  will  be  developed  prior 
to  DSARC  II  and  will  be  maintained  throughout  the 
life. 

* Support  items  required  to  develop  and  maintain 
computer  resources  will  be  specified  as  deliver- 
able . 

* Milestone  definition  and  attainment  criteria  that 
applies  to  system  and  support  software  will  be 
utilized  throughout  the  development. 

* DoD  approved  High  Order  Programming  Languages 
(HOL)  will  be  used  to  develop  Defense  system 
software . 

In  addition,  the  DoD  Software  Steering  Committee  estab- 
lished the  DoD  Defense  System  Software  Management  Program  whose 
objective  isi 

to  devise  and  carry  out  a comprehensive  and  integrated 

solution  to  the  problem  of  embedded  commuter  system 

resource  acquisition,  management  and  use,  (5) 

Specific  elements  of  the  objective  are  (1)  to  establish 
engineering  discipline  and  rigor,  (?)  to  promote  management 
visibility,  (3)  to  improve  cost  and  schedule  control,  to 
determine  the  best  methods  for  improving  software  quality  (5). 

Cn  16,  17  June  1976,  the  Defense  Systems  Management  College 
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(DSMC)  conducted  a workshop  on  the  management  of  software 
acquisition  for  embedded  computer  systems.  The  workshop 
focused  on  management  issues  encountered  at  the  program 
manager  level.  Representatives  from  all  services  and  OSD 
were  in  attendance  and  their  background  and  current  involve- 
ments provided  user,  developer  and  logistician  insights 
into  the  discussions  and  issues.  The  findings  and  recommenda- 
tions of  the  workshop  can  be  summarized  as  follows  (7) 

* Lack  of  discipline  in  specifying  and  validating 
requirements. 

Require  comprehensive  analyses  of  system 
requirements  and  defined  software  performance 
specification  prior  tc  Full  Scale  Development. 


Delay  in  the  development  of  support  software  | 

leads  to  delay  in  the  operational  readiness  of  ] 

embedded  computer  system,  ] 

Include  individuals  knowledgeable  of  software  | 

1 


embedded  computer  system,  particularly  the 
need  for  centralized  support  facilities  for 
systems  that  are  operationally  integrated, 

* There  is  a shortage  of  software  engineers  in  the 
military  and  civil  service. 

Establish  appropriate  specialty  codes  for 
software  engineers  and  develop  paths  and 
incentives  to  attract  and  retain  software 
engineers. 

* There  are  many  documentation  standards  for  soft- 
v'are  within  DoD. 

The  Joint  Logistics  Commanders'  Software 
Reliability  Work  Group  should  undertake  the 
development  of  a DoD  standard  for  software 
d ocument a t i on. 

3.  Software  Development  Phases 

Computer  software  development  can  be  grouped  into  definite 
sequential  phases  which  occur  at  least  once,  but  in  the  usual 
case  will  be  repeated  many  times  in  a system  life  cycle  as 
major  changes  in  system  requirements  occur.  This  dynamic 
environment  seems  to  be  a reality  and  if  software  development 
is  going  to  be  successful  in  terms  of  cost,  schedule  and 
performance,  it  must  be  dealt  with  in  the  management  process 
of  software  acquisition.  This  does  not  mean  that  change 
should  be  accepted  without  challenge  but  rather  that  flexibil- 
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ity  should  be  designed  into  the  software  acquisition  process. 
We  cannot  stop  the  dynamic  nature  of  weapon  systems  that  are 
built  on  the  leading  edge  of  technology  and  we  cannot  deny 
that  one  of  the  underlying  attributes  of  computer  software  is 
its  flexibility.  However,  this  realization  must  be  made  at 
the  outset  in  order  that  a responsive  software  development 
can  be  envisioned  and  implemented. 

3. 1 Requirements  Definition 

This  phase  of  software  development  is  probably  the  number 
one  culprit  that  adversly  affects  the  cost,  schedule  and 
performance  of  a computer  program  development.  The  ability  to 
specify  the  requirements  of  a software  element  in  a weapon 
system  has  been  lacking.  It  is  the  opinion  of  the  author  that 
two  m.ajor  factors  are  responsible  for  this  deficiency.  First, 
the  prim.e  attribute  of  computer  software,  flexibility,  has 

A 

given  cause  to  such  opinions  as»  let's  allocate  this  function 
to  a computer  program  and  we  can  define  it  later.  After  all, 
a computer  can  be  programmed  to  do  most  anything.  That  last 
statement  is  probably  true  but  you  may  not  be  willing  or  able 
to  pay  the  price  in  terms  of  cost  and  schedule.  The  subset  of 
this  argument  is  the  approach  often  taken  which  is  to  get  the 
program  working  and  document  later.  This  should  be  cause  for 
administering  a fate  worse  than  death,  ie,  providing  support 
for  somebody  else's  un-doc urn ented  computer  program. 

The  second  factor  which  relates  to  the  inability  to 
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accurately  specify  the  requirements  of  a computer  program  is 
the  reluctance  to  utilize  the  software  experts  in  the 
definition  process.  This  should  be  a ground  rule  for  the 

staffing  requirement  of  a PMO  involved  in  a weapon  system  i 

making  use  of  computer  programs.  By  software  experts,  we  are 
talking  in  terms  of  both  operational  and  support  functions. 

This  would  include  the  facilities,  tactical  programs,  simula- 
tion programs,  test  programs  and  procedures  and  the  software 
support  activity.  Only  through  direct  involvement  in  the  i 

.4 

requirements  definition  phase  can  these  types  of  experts  map  ’ 

realistic  approaches  to  an  efficient  software  development. 

The  software  functional  requirements  should  be  derived  through 
analysis  and  tradeoffs  of  the  operational  requirements  for 
the  system  or  modification  to  the  system.  Requirem.ents  should 
be  formulated  from  the  analysis  in  terms  of  defining  preferred 

■I 

configuration  and  design  approaches  as  part  of  the  systems  1 

engineering  process.  This  software  requirements  baseline  is  ; 

j 

normally  identified  and  controlled  as  a Program  Performance 

Specification  (PPS)  which  is  submitted  for  approval  at  the  * 

i 

Preliminary  Design  Review  (PDR).  1 

3 

3*2  Program  Design 

Following  approval  of  the  PPS,  the  next  phase  of  the 
software  development  is  the  development  of  a design  approach 
based  on  mathematical  models,  functional  flow  charts  and 
supportive  analysis  and  testing  at  several  levels.  In  essence. 
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the  prosnm  architecture  and  test  requirements  should  be 
formulated.  Allocation  of  functions  to  modules  or  subprograms 
and  associated  interface  requirements  are  established  in  this 
phase.  Testing  methodology  should  be  established  that  is 
sufficient  to  ensure  successful  attainment  of  performance  at 
various  levels  of  design  completion.  In  his  book  "Techniques 
of  Program  Structure  and  Design" , Mr.  Edward  Yourdon  effective- 
ly relates  the  interdependence  of  the  various  functions  of 
program  development.  Although  primarily  addressed  to  the  ADP 
world,  the  basic  program  developmient  considerations  discussed 
in  the  book  apply  equally  as  well  to  embedded  weapon  system 
com.puter  programs.  In  regards  to  program  design  and  the 
effect  on  subsequent  phases  of  program  development, 

the  most  logical  way,  and  in  some  cases,  the  only 

way,  to  make  a program  easy  to  test,  easy  to  maintain, 
easy  to  upgrade,  and  easy  for  som.eone  else  to  take  over 
is  to  keep  it  simrle  and  straight-forward  (15«2?) 

At  this  point,  decisions  must  be  made  concerning  the  software 
design  aprroach;  ie,  top-down,  bottom-up,  top-down  design/ 
bottom-up  implementation,  etc.  This  decision  does  not  appear 
to  be  critical  in  terms  of  implem.entation  success  but  is 
critical  in  terrr.s  of  test  philosophy.  Testing  that  will  occur 
at  both  the  module  level  and  system  level  will  necessarily 
depend  on  the  design  approach  and  system  requirements.  The 
results  of  the  design  phase  is  normally  the  Program  Design 
Sreci ‘‘ication  (PDS)  which  is  submitted  for  review  ^nd  approval 

la 
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at  the  Critical  Design  Review  (CDR). 


3.  3 Imrle-ienta  tion 

Following  approval  of  the  PDS,  the  number  crunching 
activity  occurs  which  converts  the  performance  and  design 
requirements  of  the  PP3  and  the  PDS  into  working  code.  J'iany 
aspects  of  implementation  could  and  should  be  discussed  in 
connection  with  the  total  software  acquisition  process.  In 
the  interest  of  scope,  only  some  of  the  more  predominant 
consideration  will  be  addressed  in  this  paper.  The  first  five 
words  of  this  paragraph  should  b=  held  near  and  dear  to  the 
heart.  Never  start  coding  until  design  has  been  defined  and 
approved.  There  has  been  progress  made  in  this  respect  in 
recent  years  but  it  is  still  the  number  one  consideration  to 
successful  implementation.  Never  turn  a room  full  of  black-art 
practitioners  loose  to  work  their  magic  until  everyone  knows 
what  is  to  be  done  and  what  is  not  to  be  done.  Use  higher 
order  languages  (HOL)  when  practical.  The  reasons  for  this 
simply  stated  are;  (1)  HOL  are  easier  to  implement  than 
assembly  l3nguag''S,  (2)  the  resultant  program  is  simplier  to 
test,  (3)  the  resultant  program  is  easier  to  change  and  (4) 
the  t nnsf-* rabi  1 ■ t‘/  of  the  program  is  maximized.  Often  HOL 
wl’.l  l»-'3  •'■'’icient  than  assembly  language  in  terms  of  core 
reja'r-  *s,  )ut  in  light  of  the  relatively  high  cost  of 
* - ' ■ ftwsre  as  compared  to  computer  hardware,  it 
3 e'Tective  tradeoff. 
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The  application  of  structured  programming  techniques  in 

recent  years  has  gone  a long  way  in  bringing  the  implementation 

phase  out  of  the  black-art  arena.  The  literature  now  contains 

structured  programming  material  and  should  be  given  serious 

attention.  Mr.  Yourdon  devotes  an  entire  chapter  to  the  theory 

and  techniques  of  structured  programming. 

the  notion  of  structured  programming  is  a 

philosophy  of  writing  programs  according  to  a set 
of  rigid  rules  in  order  to  decrease  testing  problems, 
increase  productivity,  and  increase  the  readability 
of  the  resulting  program.  The  primary  technique  of 
structured  programming  is  the  elimination  of  the 
GO-TO  statement  and  its  replacement  with  a number 
of  other  well-structured  branching  and  control 
statements.  (15*14^^) 

One  must  be  careful  not  to  let  the  pendulum  swing  too  far 
in  the  opposite  direction.  There  is  probably  some  mid-stream 
approach  between  the  un-disciplined  black-art  approach  and  the 
disciplined  structured  programming  approach  that  will  win  out 
in  the  end.  Given  the  choice  of  the  two,  the  structured 
programming  approach  seems  to  more  closely  fit  the  total 
acquisition  process  from  a technical  and  management  point  of 
view. 

During  the  implementation  phase,  the  Program  Description 
Document  (PDO)  and  Data  Base  Document  (D3D)  are  geiierated 
which  provide  disclosure  of  the  program  design  and  implement- 
ation. In  addition,  the  test  procedures  and  results  of  the 
module  testing  (debugging)  should  be  documented  and  retained 
for  historical  reasons.  During  this  phase,  module  integration 
and  system  integration  Test  Plans  and  Procedures  (TPF)  should 
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be  generated. 

3.4  Module  Inte,^ration 

Up  to  this  point  testing  of  the  implementation  and  design 
has  been  primarly  an  individual  process  involving  the  person 
directly  responsible  for  generating  the  code.  The  modules 
will  have  been  debugged  to  insure  proper  implementation  of  the 
design  intent.  This  often  involves  the  use  of  static  test 
generators  which  allows  the  programmer  to  ascertain  that  for  a 
predefined  set  of  data,  the  module  is  capable  of  providing ^an 
expected  response.  In  the  module  integration  phase,  the 
emphasis  is  on  combining  individual  modules  into  larger  and 
larger  program  elements.  At  this  point,  testing  should  be 
accomplished  in  accordance  with  an  approved  TPP  and  should  be 
conducted  by  a group  or  person  whose  primary  function  is  test. 
The  individual  completed  program  nodules  would  have  been  placed 
under  configuration  control  and  any  changes  to  the  implementa- 
tion or  design  of  an  individual  module  would  be  considered  in 
terms  of  system  impact.  This  is  a crucial  phase  in  the  program 
development  cycle.  Any  program  errors  (bugs)  that  survive  past 
this  point  will  be  more  difficult  to  find  and  correct  in  the 
future.  Therefore,  we  must  recognize  the  necessity  of  extens- 
ively verifying  proper  program  operation  before  it  is  made  part 
of  an  integrated  hardware  and  software  environment.  Obviously 
we  cannot  be  completely  certain  of  program  operation  until  it 
is  tested  with  the  total  system  but  all  possible  risks  should 
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be  minimized.  Mr.  Yourdon  makes  the  observation, 

from  a Philosophical  point  of  view,  you  must 
remember  that  if  you  write  a program  so  clever, 
so  intricate,  and  so  complicated  that  only  you 
can  understand  it,  then  the  program  is  worthless t 
if  your  program  makes  use  of  undocumented,  esoteric 
features  of  the  (computer)  hardware,  it  is  worthless! 
if  your  program  is  not  commented  and  documented,  it 
is  worthless.  All  of  these  practices  lead  to  a 
worthless  program  because  they  make  it  much  more 
difficult  to  test.  As  a result,  the  program  becomes 
unnecessarily  expensive  in  terms  of  time  and  money. 

(15i8) 

3. 5 System  Integration 


In  the  system  integration  phase  the  computer  program  is 
tested  as  an  integral  part  of  the  total  system.  This  is 
normally  a progressive  process  involving  different  levels  of 
functional  performance.  In  a weapon  sys'^em  development  this 
should  be  a highly  structured  procedure  in  which  the  functional 
allocations  resulting  from  the  system  engineering  process 
establish  the  testing  criteria.  During  this  phase  the  computer 
program  will  tend  to  lose  its  identity  as  it  becomes  an  embedded 
part  of  the  weapon  system.  This  is  a natural  occurrence  and 
should  be  recognized  as  such.  Tests  should  be  designed  to 
verify  system  performance  that  includes  all  the  functions 
that  have  been  allocated  to  computer  programs  and  hardware. 

In  cases  where  actual  system  hardware  is  not  available, 
simulations  are  often  used  to  complete  the  system  environment. 

In  addition,  systems  utilizing  computer  programs  have  the 
inherent  capability  of  extracting  critical  data  parameters 
that  can  be  used  for  evaluating  system  performance.  Data 
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reduction  prosrrarns  are  required  to  convert  the  extracted  data 
into  a meaningful  form  for  analysis.  These  types  of  support 
programs  should  be  managed  and  developed  throughout  the 
acquisition  cycle  as  deliverable  items  of  the  system.  They 
will  also  be  required  in  the  software  support  phase  and 
therefore  should  be  fully  documented  and  certified.  Consider- 
ation of  support  software  should  be  made  early  in  the 
acquisition  process,  which  is  often  not  done  due  to  fiscal  and 
schedule  constraints.  In  analogy  to  hardware,  it  would  be 
unthinkable  to  field  a missile  launcher  without  providing 
the  necessary  support  equipment  to  maintain  it. 

3.6  Support 

The  final  phase  in  the  software  development  cycle  is 
most  often  termed  computer  program  maintenance.  This  appears 
to  be  a carry  through  from  hardware  technology.  Unlike 
hardware,  a digital  computer  program  does  not  break  and  it 
does  not  degrade  and  if  it  ever  worked  it  will  continue  to 
work  for  a given  set  of  requirements.  For  this  reason,  the 
final  phase  would  more  accurately  be  described  as  the  support 
phase,  During  this  period  the  bugs  that  survived  the  module 
and  system  integration  phases  start  to  come  out.  Therefore 
one  activity  of  this  phase  is  the  removal  of  program  errors 
that  evaded  the  earlier  testing.  Secondly,  the  probability 
of  requirements  remaining  constant  throughout  the  life  of  a 
weapon  system  is  zero.  Therefore  the  capability  to  modify 
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the  computer  programs  to  meet  the  new  requirements  must  be 
retained.  The  two  types  of  activities  mentioned  above  are 
not  a complete  discription  of  the  support  phase  but  do  point 
out  the  need  for  managing  and  developing  support  software  as 
deliverable  items. 

The  six  phase  software  development  cycle  described  above 
is  based  on  the  author's  experience  and  study.  It  is  not 
presented  as  the  one  and  only  way  to  develop  software.  It 
does  provide  a reasonable  approach  to  the  development  of 
embedded  computer  programs  for  the  weapon  system.  In  general, 
the  first  two  phases  of  requirements  definition  and  program 
design  should  represent  about  ^0%  of  the  total  development 
effort  when  the  final  phase  of  support  is  excluded  based  on 
its  continuing  nature.  Implementation,  which  includes  the 
coding  and  debugging  of  individual  modules,  would  represent 
about  20%  of  the  total  effort  and  the  module  and  system 
integration  represents  the  remaining  kOfo,  In  an  even  more 
general  sense,  the  software  cycle  can  be  correlated  to  the 
weapon  system  acquisition  cycle.  The  requirements  definition 
and  program  design  phases  would  normally  occur  in  the  Conceptual 
and  Validation  phases  of  weapon  system  acquisition.  In  most 
cases  however,  the  Validation  phase  will  demonstrate  the 
feasibility  of  a system  design  through  actual  implementation 
of  intermediate  levels  of  hardware  and  software  configurations. 
In  such  cases,  some  portions  of  the  first  five  software 
development  phases  will  occur.  This  can  be  considered  as  a 
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subset  or  a-lvanced  development  of  the  total  system  develop- 
ment. During  the  Full  Scale  Development  phase  the  implementa- 
tion, module  integration  and  system  integrations  activities 
would  occur.  This  leads  to  a complete  validation  of  the 
software  that  is  to  be  part  of  the  system. 

In  addition  to  the  software  development  cycle  discussed 

above,  various  other  approaches  can  be  found  in  the  literature. 

In  Philip  Metzger's  book , "Managing  a Programming  Project"  a 

six  phase  process  is  described  (11) 

Definition  - a project  plan  is  written  and  the  technical 
problem  is  defined t solutions  are  deferred. 

Design  - a design  document  is  written  which  describes 
an  acceptable  solution  to  the  problem. 

Programming  - build  and  test  a program  system  according 
to  the  solution. 

System  Test  - separate  group  test  the  program  in  a 
"live"  environment . 

Acceptance  - finished  program  system,  including 
documentation,  is  demonstrated  to  the  customer,  and 
tested  against  acceptance  criteria  mutually  agreed  to 
earlier  in  the  development  process. 

Installation  and  Operation  - programs  are  introduced 
and  tested  on  customer's  equipment  in  the  ultimate 
operating  environment, 

Mr.  Metzger  provides  a thorough  development  of  all  aspects 
of  computer  programining  and  software  management  and  is  highly 
recommended  for  all  software  managers. 

Software  Costs 

The  concern  that  has  been  expressed  in  recent  years  at 


all  levels  of  DoD  in  regards  to  software  development  costs  have 
probably  been  based  more  on  estimate  errors  than  on  cost- 
effectiveness  issues.  Cost  estimating  for  software  develcp- 
ment  is  a subject  deserving  of  an  in-depth  treatment  and  will 
be  addressed  in  this  paper  in  a general  manner.  For  additional 
information  on  the  subject  the  reader  is  referred  to  Defense 
Systems  Management  School  study  reports  prepared  by  A.  W. 

Andres  (2)  and  R.  A.  Findley  (9), 

Cost  estimate  errors  in  software  development  programs 
appear  to  be  directly  related  to  the  management  approach.  It 
has  only  been  in  recent  years  that  adequate  attention  has  been 
directed  to  software  acquisition.  As  a result  we  have  been 
moving  along  the  learning  curve  and  paying  our  dues  in  cost, 
schedule  and  performance.  Several  major  factors  that  have 
negatively  influenced  the  accuracy  of  cost  estimates  are  also 
the  targets  of  the  management  improvement  efforts.  Of  prime 
importance  is  the  definition  of  reo^uiremients  for  computer 
programs.  We  must  achieve  the  ability  to  accurately  specify 
software  requirements  if  we  ever  hope  to  accurately  assess 
costs.  These  requirements  must  include  the  tactical  or 
operational  functions,  the  support  software  functions, 
documentation  and  facilities.  Application  of  system  engineer- 
ing concepts  and  software  expertise  in  the  early  concept 
formulation  of  software  requirements  is  one  method  of  improving 
this  ability.  All  aspects  of  software  development  must  be 
considered  when  arriving  at  cost  estimates. 
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Once  requirernents  have  been  defined,  complete  and  detailed 
planning  must  follow  which  establishes  the  methodology  of 
implementing  a design  which  will  meet  the  requirements. 

Proper  generation  of  the  Program  Performance  Specification 
(PPS),  and  Program  Design  Specification  (PDS)  and  Test  Plans 
and  Procedures  (I'PP)  will  identify  the  scope  of  the  tasks  to 
be  accomplished.  From  this  point  further  planning  should 
occur  in  the  form  of  Work  Breakdown  Structures  (WBS),  Network 
Diagrams  and  Flow  Charts  which  identify  the  tasks  to  be 
accomplished  in  individual  definable  units  which  can  be 
accurately  scheduled  and  priced.  In  general  an  experienced 
programming  organization  can  make  accurate  estimates  on 
definable  segments  of  code.  The  real  problem  lies  in  arriving 
at  that  level  of  definition  and  holding  it  constant. 

Two  forms  of  cost  estimating  will  normally  be  required. 

In  the  initial  requirements  formulation  period,  historical 
analogies  will  serve  as  the  data  base  for  making  estimates. 
These  are  the  estim.ates  that  have  been  most  prone  to  error 
due  to  the  lack  of  corporate  memory  resulting  from  the  short 
history  of  weapon  system  software  management.  As  we  improve 
our  management  techniques,  our  cost  data  b^^se  should  improve. 
The  management  approach  must  be  disciplined  to  account  for 
all  aspects  of  software  development,  especially  in  the  support 
programs  and  documentation  areas.  Secondly,  the  inherent 
flexibility  of  software  must  be  prudentially  applied.  If 
functions  are  planned  in  the  earlier  periods  that  are  allocated 
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to  software  flexibility,  the  cost  estimates  are  sure  to 
suffer. 


The  second  form  of  cost  estimating  which  should  be 
applied  in  development  process  is  based  on  detailed  breakout 
of  software  tasks.  Once  this  level  of  definition  is  achieved 
through  WBS  and  the  like,  accurate  estimates  can  be  achieved. 


SECTION  III 


SOFTWARE  MANAGEMENT  IN  A WAJOR  NAVY  ACQUISITION  (AEGIS) 

1.  Prelude 

Up  to  this  point  in  the  report,  the  intent  has  been  to 
examine  the  current  trends  in  computer  program  management 
based  on  an  analysis  of  related  publications  and  the  author's 
experience.  In  this  section  we  will  focus  our  attention  on  a 
real  world  application  of  computer  program  management  in  a 
major  weapon  system  acquisition.  No  attempt  will  be  made  to 
provide  an  exhaustive  analysis  of  the  management  approach  or 
the  technical  design  considerations  incorporated  in  the  weapon 
system  acquisition.  Instead  our  attention  will  be  directed 
toward  understanding  the  computer  program  management  concepts 
that  are  being  employed.  The  choice  of  the  AEGIS  Weapon 
System  for  this  exercise  was  based  on  the  currency  and 
complexity  of  the  computer  program  contribution  to  the  total 
integrated  weapon  system. 

The  information  contained  in  this  section  has  been 
compiled  from  the  AEGIS  Combat  System  Computer  Program 
Development  Plan  (1),  the  Applied  Physics  Laboratory,  John 
Hopkins  University  DoD  Weapon  Systems  Software  Management 
Study,  (3  & 4)  and  through  interviews  conducted  with  personnel 
in  the  AEGIS  Program  Manager  Office  (PMS-403)  and  in  the  AEGIS 
Lead  Laboratory  Office  (Naval  Surface  Weapons  Center,  Dahlgren 
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Laboratory) . 


2 . SysteTT'  Description 

The  AEGIS  Weapon  System  is  an  integrated  shipboard 
detection,  command  and  weapon  control  system  that  is  being 
designed  for  installation  in  a wide  variety  of  ship  classes. 

It  is  a fast-reaction,  high  performance  Weapon  iiystem  engineer- 
ed to  provide  the  Fleet  with  a wide  area  surface-to-air  and 
surface-to-surface  defense  through  the  1980's  and  beyond. 

When  used  with  long  range  surface-to-surface  and  extended 
range  surface-to-air  missiles  the  system,  will  provide  the 
Navy  with  a major  offensive  surface  strike  capability. 

An  AEGIS  ship  Combat  System,  of  which  the  AEGIS  Weapon 
System  in  an  integral  part,  consists  of  twenty-three  separate 
elements,  ten  of  which  recuire  computer  programs.  The  major 
elements  of  the  AEGIS  Weapon  System  include  a multifunction 
phase-phase  array  Radar  System  (AN/SPY-lA) , Weapon  Control 
System  (WCS  Kark  1),  Fire  Control  System  (FCS  Mark  99),  Command 
and  Decision  System  (C&D  Mark  1),  Operational  Readiness  Test 
System  (CRTS  Mark  1),  Guided  Missile  Launching  System  (Mark  26), 
and  the  Standard  Missile-Medium.  Range  (SM-2/MR).  The  first 
five  of  these  elements  contain  embedded  computer  programs 
which  are  being  developed  by  the  AEGIS  Weapon  System  prime 
contractor  with  participating  sub-contractors.  Computer 
pro^ams  for  the  elements  outside  of  the  AEGIS  Weapon  System 
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are  beinp-  developed  separately  by  Navy  Acquisition  Managers 
(NAM's).  All  embedded  computer  programs  will  be  integrated 
into  the  Com.bat  System  by  the  AEGIS  Weapon  System  prime 
contractor. 

The  AEGIS  system  is  fully  automated  and  includes  the 
latest  Naval  Tactical  Data  System  (NTDS)  and  makes  wide  use 
of  the  standard  AN/UYK-7  and  AN/UYK-20  digital  computers. 
Modular  system  design  was  employed  which  is  adaptable  to  a 
variety  of  ship  hulls.  System  development  includes  the 
construction  of  two  Engineering  Development  Models  (EDM's) 
prior  to  implementation  in  the  first  operational  ship.  The 
first  SDK  has  been  completed  and  is  currently  aboard  USS  NORTON 
SOUND.  It  is  a limited  performance  system  intended  to  provide 
verification  of  critical  AEGIS  capabilities.  The  second  EDM, 
which  represents  a proto-type  of  the  operational  system,  will 
be  installed  and  tested  in  a landbased  Combat  System  Engineer- 
ing Development  Site  (CSEDS).  The  purpose  is  to  provide 
verification  of  system  engineering,  interfaces,  ship  design 
support,  and  operational  computer  programming. 

Computational  requirements  in  the  AEGIS  Weapon  System  are 
complex  and  critical  to  system  performance.  The  Radar  System 
which  conducts  area  search,  automatic  target  detection  and 
tracking,  and  provides  midcourse  guidance  communication  with 
the  SM2  missile,  is  controlled  by  a four-bay,  memory-shared 
suite  of  AN/UYK-7  computers  consisting  of  256,000,  bit 
memory  loc'»tions.  An  identical  AN/UYK-7  computer  configuration 


is  used  in  the  C<iD  System  to  provide  tactical  decision 
support,  multisensor  data  correlation  and  management,  air 
intercept  control,  data  link  with  other  units  and  various 
display  control  functions.  A third  identical  AM/UYK-7  computer 
suite  is  employed  in  the  WCS  for  weapon  assignment  and 
scheduling.  In  addition  to  the  above  mentioned  A^/UYK-7 
computers  there  are  also  several  of  the  smaller  scaled  standard 
AN/UYK-20  computers  used  throughout  the  System.  Comiputer 
progra'"3  that  are  being  developed  fall  into  one  of  three  broad 
categories.  Operational  Programs  consist  of  the  Radar  System, 
Vi'CS,  PCS,  C&D,  CRTS  and  Training  programs.  The  Executive 
Program  category  contain  the  AEGIS  Tactical  Executive  System 
(ATES)  and  the  Standard  Executive  for  the  AN/UYK-20  (SDEX/2C). 
The  third  category  contain  a host  of  Support  Programs  such  as 
the  standard  Compiler  Monitor  System  (CMS-2),  AEGIS  Development 
Operating  System  (ADCS),  Interface  Simulator  System  (ISS)  and 
the  AEGIS  Data  Reduction  (ADAR)  program. 

In  a system  such  as  AEGIS,  the  above  description  is  at 
best  the  tip  of  the  ice-berg  in  regards  to  the  considerations 
that  are  being  addressed  by  the  computer  program  management 
function.  The  viev;  held  by  the  PK  office  that  computer 
program  developm.ent  will  be  managed  as  an  integral  part  of  the 
system  as  opposed  to  a separate  entity  and  the  following 
realization  found  in  the  AEGIS  Combat  System  Computer  Program, 
Development  Plan  in  essence  sum.s  the  approach  being  taken: 
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It  is  fully  recogni?ed  thst  a development  program  of 
this  magnitude  and  complexity  cannot  be  undertaken 
without  assuming  some  inherent  risk.  The  objective 
of  this  plan  is  to  structure  a discipline!  and 
orderly  program  that  will  reveal  problems  sufficient- 
ly early  to  minimize  long-term  risk.  Fart  of  this 
plan  includes  the  3uild-a-Little , Test-a-Little , 
Integrate-a-Little  concept  to  assure  an  orderly, 
efficient  computer  program  development  and  integration 
process  (1»5) 

In  discussions  with  the  PyO,  it  was  pointi^d  out  that  specified 
Reaction  Time  in  the  Anti-Aircraft  Warfare  (AAW)  mode  is  the 
overwhelming  and  dominant  requirement  that  drives  all  parts  of 
the  computer  program  development. 

3.  System  Acquisition  History 

The  Advanced  Surface  fissile  System  was  born  In  concept 
i.n  196 3«  Design,  development  and  demonstration  of  essential 
elements  of  the  multifunction  array  radar  were  accomplished 
by  APL/JHU  and  provided  a firm  basis  for  full  Scale  Develop- 
ment of  the  radar.  In  April  of  I96S,  the  Secretary  of  Defense 
signed  Development  Concept  Paper  16  which  resulted  in  DSAhC  I 
approval  for  initiation  of  Contract  Definition.  In  I969i  RCA 
was  awarded  a contract  for  the  Engineering  Development  of  the 
AEGIS  Weapon  System,  DSARC  II  approval  was  obtained  in  June 
1974  which  supported  acquisition  of  the  AEGIS  System.  As 
noted  in  the  AFL  study,  early  consideration  was  given  to 
computer  program  development! 

During  the  initial  engineering  phase,  program  decisions 
were  made  to  develop  a functionally  modular  computer 
program  and  provide  a Tactical  Executive  Program 
structured  for  AEGIS.  The  specificati ons  and  planning 
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reflecterl  strong  eirnhasis  on  the  software  acquisition 
■procurements  (4i6-2) 


4.  Computer  Program  Definition.  Design  and  Implementation 
4. 1 Definition 

Mission  requirements  contained  in  the  Development  Concept 
Paper  (DCP),  Tactical  Operational  Requirements  (TOR)  and  the 
Top  Level  Requirements  (TLxR)  are  translated  into  specific 
performance  requirements  for  the  AEGIS  System  and  are  document- 
ed in  the  AEGIS  System  Specification.  The  first  stage  of 
Functional  Allocation,  which  relates  to  computer  program 
development,  is  the  generation  of  the  prime  system  elements 
specifications.  Further  Functional  Allocation  occurs  as  the 
prime  system  elements  requirements  are  allocated  to  the 
specific  equipment  and  program  subsystems  constituting  each 
system  element.  r.iIL-SID-490  (12),  augmented  by  SSCNAVIimST 
2560.1(14)  for  the  computer  program  specifications,  governs 
the  hierarchy  and  f orma of  specif  ica  tions  which  document  the 
total  system  design. 

A tool  developed  in  the  AEGIS  Project  for  the  allocation 
of  system-level  function  to  subsystems  is  the  Functional  Flow 
Diagrams  and  Descriptions  (F-D^).  Documents  are  produced 
under  the  F-D^  process  at  the  systea  level  by  functional 
analysts  with  the  support  of  sys  tetn  designers.  In  the  APL/JHU 
study  the  F^D^  process  was  described  as  follows  1 
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It  transl'ites  the  requirements  of  the  AEGIS  Weapon 
System  into  hierarchically  ordered  functional  block 
diagrams  and  associated  textual  descriptions  at 
every  level  of  system  operation.  Inputs  to  each  block 
and  their  sources  are  identified  as  are  outputs  and 
their  destinations.  Reouired  functions  are  indicated 
and,  at  lower  levels,  allocated  appropriately  to 
computer  programs,  equipment  items,  or  operator 
stations.  At  every  level,  traceability  is  provided 
to  higher  and  lower  levels.  At  the  lower,  more 
detailed  levels,  direct  reference  is  made  to  specific 
paragrarhs  within  specification  and/or  design 
documents.  This  enables  a complete  system  audit  by 
making  all  functions  tracable  to  approved  documentation 
and  all  documentation  tracable  to  allocated  functions. 
(4i6-50) 


The  definition  of  the  computer  program  requirements 
begins  at  -^he  Tier  2 level  of  the  which  defines  the 

individual  element  requirements  and  the  element-to-element 
functional  interfaces.  Inherent  to  this  comprehensive  system 
engineering  approach  is  the  early  consideration  of  computer 
program  development  as  an  embedded  part  of  the  system  develop- 
ment. As  functions  are  allocated  to  either  equipment  or 
computer  programs  at  the  system  element  level,  equipment/ 
program  tradeoffs  are  of  major  concern. 

Once  the  system  elements  functions  have  been  allocated  to 
a computer  program,  the  definition  of  the  perfromance  require- 
ments for  the  program  drive  the  generation  of  the  Computer 
Program  Performance  Specification  (PPS)  for  the  element.  The 
PPS  is  generated  by  the  systems  engineering  activity  with 
assistance  from  the  programming  activity.  Further  development 
of  th°  F^D^  to  the  Tier  3 level  provides  further  definition 


of  the  function  allocation  to  a level  sufficient  for  detailed 
program  design.  Various  tradeoff  studies  and  supporting 
analysis  are  conducted  at  this  point  to  demonstrate  the 
feasibility  of  achieving  the  required  performance  with  a given 
design  approach.  This  material  along  with  the  preliminary  PFS 
is  presented  to  the  Navy  for  a Preliminary  Design  Review  (PDR) 
of  the  element  computer  program.  Approval  of  the  PPS  estab- 
lishes the  allocated  baseline  for  the  computer  program 
performance  and  is  the  basis  for  further  development  of  the 
computer  program. 

4. 2 Design 

At  this  point,  program  functions  are  allocated  to  sub- 
levels  in  the  program.  Functional  description  of  the  modules, 
module  interfaces,  core  and  timing  estimates,  program  control 
logic  and  other  considerations  of  the  computer  program 
architecture  form  the  basis  for  generation  of  the  Computer 
Program  Design  Specification  (PD3),  Following  an  iterative 
refinement  process,  a draft  version  of  the  PDS  is  reviewed  at 
a Preliminary  Critical  Design  Review  (PCDR).  Completing  this 
milestone  in  the  computer  program  development  process  starts 
the  generation  of  the  final  PDS  and  the  Build  process.  This 
activity  is  characterized  by  the  generation  of  the  preliminary 
Program  Description  Document  (PDD)  and  the  Data  Base  Design 
(DBD)  and  the  Build  In-Progress  Review  (IPR's).  The  term 


Build  refers  to  a partial  or  complete  computer  program  that 
can  execute  several  or  all  computer  program  functions. 

Processing  within  Builds  which  are  not  available  is  simulated 
with  core  and  time  stubs  in  order  to  maintain  realism  in  the 
operational  environment.  As  Builds  are  coded  and  tested,  they 
become  part  of  a more  encompassing  Build  and  eventually  part 
of  the  total  program.  Following  the  Build  IPR,  the  program 
Build  is  released  for  code  and  test.  Throughout  the  definition  j 

and  design  process,  interfaces  between  the  computer  programs 
and  equipment  are  control  via  computer  program  interface 
documentation. 

Since  operational  functions  and  modules  do  not  always 
relate  directly,  a top-down  design  technique  has  been  used 
that  involves  the  use  of  System  Verification  Diagrams  (SVD) 
and  threads. 

SVD's  illustrate  the  inputs,  conditions,  tasks,  and 
outputs  constituting  an  element  computer  program 
functional  allocation.  SVD's  may  be  developed 
first  at  the  most  general  level.  Subsequent 
progressive  analysis  furnish  the  more  det‘=iled 
(subfunction)  levels  until  each  SVD  represents 
a group  of  related  tasks  in  terms  of  computer 
program  logic  (algorithm,  procedures,  decisions, 
etc).  Saco  such  t^sk  is  then  represented  as  a 
thread,  which  represents  a flow  of  data  through 
various  process  steps  from  stimulus  through  response. 

(10-9) 

A major  characteristic  of  the  above  design  approach  is  its 
supportive  nature  to  the  verification  and  validation  of  the 
computer  program  as  an  integral  component  of  a system  element. 
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In  addition  it  provides  a ready  means  of  evaluating  the  impact 
of  change  in  system  or  element  requirements.  The  Critical 
Design  Review  (CDR)  marks  the  formal  review  and  approval  of 
the  Computer  Program  Design  Specification  (FDS)  which  represents 
the  allocated  program  design  baseline. 

4, 3 Implementation 

Once  the  Build  IPR  has  been  completed  the  partial 
subprograms  which  constitute  the  Build  are  released  for  coding 
which  utilizes  the  CMS-2  Higher  Ordered  Language  (HOL)  and 
structured  programming  techniques.  Builds  have  been  designed 
to  segment  the  programs  into  manageable  sized  tasks  which 
represent  a defined  low-level  subfunction.  As  coding  and 
testing  progresses,  the  Builds  are  linked  together  with 
additional  subprograms  to  perform  major  functions  associated 
with  an  element.  The  Build  evolution  is  functionally  defined 
to  establish  an  orderly  growth  in  capability  which  eventually 
interfaces  with  other  elements  of  the  system.  The  build  and 
test  process  has  been  an  effective  approach  in  identifying 
and  correcting  coding  errors  in  the  early  stages  of  program 
development . 

5.  Computer  Program  Management 
5 . 1 Organization 

The  management  function  of  the  computer  program  develop- 
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TTient  in  the  AEGIS  System  is  performed  by  the  Program  Manager 
Office  (PK3-403)  and  the  prime  contractor  (KCA)  for  the 
system  development  as  depicited  in  Figure  1.  Support  is 
provided  to  the  PMO  by  various  other  Navy  and  industrial 
organizations.  In  addition  to  the  Defense  Contract  Administra- 
tion Services  (DCAS)  Office,  the  PMO  maintains  a Technical 
Representative  Office  at  the  prime  contractor's  facility. 
Computer  system  specialists  are  included  in  the  PMO  staff  and 
at  the  contractor’s  facility  as  part  of  the  Technical 
Representative  Office  in  an  effort  to  focus  expert  attention 
on  the  computer  program  development  throughout  the  system 
acquisition  process.  As  noted  in  PMO  interviews,  the  emphasis 
is  placed  on  developing  computer  programs  as  an  integral  part 
of  the  total  system  development.  In  AEGIS  this  concept  is 
of  particular  importance  in  light  of  the  extensive  use  made 
of  computer  programs  in  meeting  the  mission  requirements. 

Within  the  prime  contractor  organization  there  has  been 
established  a multidisciplined  computer  program  development 
team  consisting  primarily  of  Navy  and  contractor  management, 
system  engineering,  design  activities,  test  and  evaluation 
and  subcontractor  management.  The  AEGIS  Combat  System  require- 
ments are  established  by  the  Combat  System  Development  Group. 
Computer  program  management,  which  is  under  control  of  a 
centralized  office,  controls  budget,  funding,  definition, 
design,  coding,  test,  integration,  configuration  management, 
a centralized  Program  Generation  Center  (PGC),  a Computer 
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Program  Test  Site  (OPTS)  and  the  Central  Computer  Program 
Library  (CCPL).  Additional  comments  concerning  the  PGC,  CPTS 
and  CCPL  will  be  made  later  in  this  report, 

5.2  Design  Review  Process 

As  in  the  management  of  any  activity  it  is  necessary  tc 
establish  methods  and  procedures,  standardization  and  feedback 
systems  that  are  capable  of  accurately  monitoring  the  status 
of  the  developm.ent  process.  In  AEGIS  a variety  of  management 
tools  has  been  emrlcyed  tc  provide  control  of  the  computer 
program  developm.ent  process.  Emphasis  has  been  placed  on  the 
definition  of  computer  programs  as  deliverable  configuration 
items  and  the  documentaticn  of  this  definition  by  a set  of 
uniform  specifications.  Requirem.ents  contained  therein  form 
the  basis  for  formal  technical  reviews  and  subsequent  test 
planning  documentaticn  and  testing.  AEGIS  uses  technical 
reviews,  Preliminary  Design  Reviews  (FDR),  Critical  Design 
Reviews  (CDR)  and  Configuration  Audit  Reviews  (CAR)  for  base- 
line management  and  contr^^ctual  milestones. 

The  FDR  is  used  as  a vehicle  wherein  all  participating 
activities,  especially  the  Navy,  review  and  approve  the 
Computer  Progra-^  Performance  Specifications  (FPS)  that  are 
written  in  accordance  with  SECNAViAST  3560,1  (6).  Reviewers 
include  the  PMC,  users,  system  engineering  activities  and 
design  activities.  At  FDR,  the  preli'iinary  design  concepts 
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are  reviewed  to  assure  that  performance  requirements  can  be 
met  within  the  constraints  of  the  equipment  and  environment 
and  that  the  com.ruter  program  performance  matches  the  perform- 
ance levied  by  the  item  specification  for  the  elements.  The 
output  of  the  PDR  process  is  the  final  draft  of  the  FPS.  It 
should  be  noted  that  the  computer  program  functional  allo- 
cations are  derived  from  the  prime  item  specification  of  the 
elem.ent  and  that  the  first  draft  of  the  PPS  is  generated  by 
system  engineering.  During  the  PDR  process,  numerous  In- 
Progress  Reviews  (IPR)  are  held  and  post  PDR  conferences  serve 
to  resolve  all  questions  concerning  the  performance  require- 
ments as  reflected  in  the  PPS. 

CDR  serves  as  the  vehicle  for  reviev?  and  approval  of  the 
Computer  Program  Design  Specification  (PDS).  The  review 
proces=!  is  basically  the  same  as  for  the  FDR  and  results  in 
formal  identification  and  approval  of  the  specific  computer 
program’^ine-  documentation  that  will  be  released  for  coding 
and  testing.  All  PDR  and  CDR  actions  are  reviewed  and 
approved  by  the  Executive  Panel.  It  should  be  noted  that 
coding  and  testing  is  not  started  prior  to  achieving  the  CDR 
milestone.  During  the  CDR  process,  work  leading  to  the  first 
Build  will  start. 

5. 3 Cost  and  Schedule  Control 

In  addition  to  normal  contractual  cost  reporting, 
provisions  have  been  made  in  the  Computer  Program  Development 
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Office  to  provide  detailed  control  and  accounting  for  costs 
associated  with  the  computer  program  development.  Work  to  be 
performed  is  broken  down  by  statements  of  work  to  the  imple- 
menting activity  and  identified  into  Work  Breakdown  Structure 
(W3S)  cost  levels.  For  each  WBS,  expenditures  are  budgeted, 
funded  and  monitored  based  on  scheduled  and  planned  accomplish- 
ment. Expense  elements  include  operation  and  maintenance  of 
the  FPG  and  CFTS  as  well  as  contractor  furnished  com.puter 
equipment. 

Schedule  control  is  accomplished  with  the  use  of  computer 
generated  network  diagrams  at  the  project,  computer  program., 
multi-elem.ent  program  and  element  program  levels.  Compatibil- 
ity of  connecting  events  and  critical  paths  are  determined  by 
the  network  generation  profrram.  Changes  and  updates  to  any 
of  the  networks  are  operator  actions  at  an  interactive  terminal. 
The  network  progra.m  automatically  generates  alerts  if  schedule 
problems  are  to  be  encountered  as  a result  of  the  nev;  informa- 
tion. 


5. ^ Verification  and  Validation 


As  defined  by  the  AEGIS  Combat  System  Computer  Program, 
Development  Flan, 

Verification  and  validation  of  the  AEGIS  computer 
programs  is  system.atic  evaluation  of  the  specifica- 
tion implementation,  and  operation  of  th''  program 
throughout  the  development  cycle.  (Ii2-1?) 

In  essence  the  verification  process  determines  that  require- 
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ments  ani  specifications  of  esrlier  stages  in  the  development 
process  have  been  translated  properly  into  more  detailed 
requirements  and  objectives.  This  process  involves  an 
iterative  reviev/  and  revise  cycle  that  translates  system 
performance  requirements  into  the  program  design  requirements. 
In  encompasses  the  FDR,  CDR  and  IPR  processes  and  all  associat- 
ed documents. 

1 While  verification  ascertains  that  the  reauirements  have 

. been  properly  allocated  to  the  computer  programs,  validation 

demonstrates  that  the  program  performs  as  required  in  its 
intended  operational  environment.  The  validation  process  is 
directly  related  to  the  Build-A-Little , Test-A-Little , 
Interrate-A-Little  philosophy.  As  defined  earlier,  the  Build 
is  a logical  grouping  of  functions  in  computer  program  modules 
or  computer  program  elements.  As  part  of  the  validation 
process.  Builds  are  tested  at  increasing  levels  of  program 
requirements  and  thereafter  becom.e  integral  parts  of  hirher 
level  Builds.  F.'.odule  or  module-part  testing  is  conducted  as 
an  Engineering  Test  & Evaluation  (ET&E)  using  informal 
program’^er  generated  test  data  and  procedure.  This  type  of 
testing  uses  program  drivers  and  the  AEGIS  Tactical  Executive 
System  (ATS3)  as  test  tools  and  are  conducted  in  Computer 
Program.  Test  Site  (CPT3).  Results  and  scenarios  of  these 
tests  are  recorded  in  the  prorrammer's  notebook.  Build  test- 
ing is  si’^ili-’r  to  module  teeming  but  represents  a validation 
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of  a "lore  complete  fro-"ra"!  requirenent.  The  Interface 
Si-^.ulator  System  (133)  proerama  v.’hich  are  certified  by  the 
developing  managers,  are  utilized  in  the  Build  testing.  As 
Build  testing  and  133  development  progresses  to  the  element 
level,  a validation  process  c-’lled  Phase  II  testing  measures 
the  element  conruter  program  test  results  against  the  FPS 
requirements.  Ph-^se  II  te=-ting  takes  pl^>ce  in  the  CPIS  using 
the  ISS  as  the  primary  t»st  tool  and  is  conducted  as  defined 
in  the  Phase  II  Test  Procedure  document  which  is  submitted  to 
the  Navy  as  a Contractual  Data  Reouirements  List  (CDHL)  item. 
The  Test  Procedure  documeiit  defines  a pass-fail  criteria  for 
each  test  ito"!  of  the  procedure.  As  element  computer  prc'ra^is 
and  equipment  integr"tion  progresses,  multielement  computer 
program  and  equipment  are  integrated  at  the  Com.bat  System 
Engineering  Development  (CSED)  Site. 


5. 5 Configuration  f.!'^nor':em.ent 


Gonf  i mura  tion  management  of  the  element  computer  pre^ra^-s 
is  maintained  throughout  the  develop- ent  cycle  via  appr-cved 
configuration  ”’anag<'-'ent  rlans.  The  level  of  control  exercis- 
ed varies  from  ninia.al  in  tha  Initial  code  ani  debug  period  to 
formal  Configuration  Control  Board  (CCB)  reviews  fol'' awing  the 
completion  of  Fha~e  II  testing.  The  PF3  and  FD3  establish  the 
specification  baseline  for  the  cor.p'utor  pro-rams  against 


'Which  configuration  is  managed.  Intermediate  configurations, 
that  relate  to  Build  plans,  are  designated  as  warkin,_  basclimes 


which  conver~e  to  th"'  opcol  f ica '^icn  conf  i-^ur"  lion 

as  cc"cuter  rro'ro’-:  ieVvary  is  . 

Conf  I i'uration  control  of  the  computer  programs  a^Jiresses 
control  of  the  changes  to  the  proarans  an1  control  of  the 
Comruter  Program  Configuration  Item  (CfCl)  specif ications  in 
relation  to  each  other  and  to  other  syste’';  re  ruirements . Two 
tools  for  performing  this  function  h.ave  been  established  in 
the  Configur“'tion  Conti'ol  Board  (CCB)  and  the  Central  Computer 
Program  Library  (CCFL),  The  CCB  reviews  and  approves  change 
packages  and  ensures  consistent  and  formally  approved 
specifications  in  accordance  with  contractual  re-^uireiaents . 

The  CCPL  objective  is  to  control  the  acauisition,  maintenance 
and  dissr^min^tion  of  approved  program  configurations.  Program 
change  control  and  history  is  mainta.ined  through  Phase  II 
testing.  Configuration  control  starts  when  the  module  has 
been  tested  and  approved  at  the  design  activity  level.  There- 
after coding  changes  can  only  occur  through  official  channels. 

Experience  on  AEGI3  EOM-1  shows  the  majority  of 
program  errors  to  be  design  related  and  not  coding 
related,  with  the  major  portion  found  during  program, 
testing.  This  multiple  build  approach  attempts  to 
find  and  correct  the  problems  early  in  pro/ram 
ievelcnment.  (4i6-41) 
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SECTION  IV 

CONCLUSICK5  AiO  R'^COl .•'END^riOKS 


1 . Conclusions 

The  primary  conclusion  that  one  iraws  from  a stuly  in 
software  -lanaijement  is  that  a vast  a'lount  of  attention  is 
beina  directed  toward  improvement.  In  general  the  improvements 
h-^ve  the  corr^on  goal  of  converting  an  art  form  into  technology 
by  arolication  of  sound  planning,  implementing  and  monitoring 
techniques.  There  is  an  overriding  objective  of  seeking  out 
a roadmap  to  successful  software  development.  One  must 
consider  the  intangible  nature  of  software  to  understand  the 
problems  that  hove  plagued  its  management.  It  is  now  realized 
that  the  powerful  black  art  cannot  be  practiced  in  a vacuum 
when  it  represents  such  a large  contribution  to  the  system  in 
v/hich  it  resides. 

This  study  has  identified  some  cornerstones  that  should 

be  considered  in  software  management. 

•? 

* , Responsibility  for  computer  progra;ns  and  equipment 

of  a par-^icular  system  or  element  should  be 
assigned  to  a single  individual. 

* Include  systems  engineering  and  soft'ware  support 
considerations  In  the  reouirements  definition  phase. 

* Assume  a realistic  attitude  tower'd  software 
flexibility. 

k? 
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Consider  support  programs  and  facilities  as 
essen'^ial  and  necessary  components  of  the  soft- 
ware development  process. 

Specify  all  software  required  for  operation  ani 
support  as  contract  deliverable  items. 

Require  complete  documentation  for  all  deliverable 
software , 

Ensure  management  visibility  through  the  use  of 
a computer  Program  Develop'^ent  Flan,  structured 
Program  Reviews  and  r>'ilestones.  Specify  attain- 
ment criteria. 

Staff  the  FKO  for  performing  technical  evaluation 
of  the  software  development. 

Use  Top-Down  design  and  modular  construction  for 
program  development. 

Develop  the  test  plan  in  parallel  with  the  program 
design.  Develop  test  procedures  in  parallel  with 
program  implementation. 

Use  High  Order  Language  (HCL)  and  Structured 
Programming  when  possible  for  i-'plenentation. 
Perform  module  integration  and  system  integration 
within  a Test  Group  separate  from  the  Programming 
Group. 

Impose  conf igura tion  control  prior  to  module 
itegration. 
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2 . Recommendations 


In  terms  of  specific  recomr^enda  tions , the  cornerstones 
of  software  development  listed  ^bove  should  be  applied  to  any 
software  acquisition.  They,  by  all  means,  do  not  represent 
an  exhaustive  treatment  of  the  management  issue  but  do  identify 
the  tyre  of  considerations  that  should  be  made  in  the  develop- 
ment process.  The  underlying  theme  of  their  use  is  to  provide 
the  definition,  design,  planning,  implementation  and  testing 
requirements  of  computer  programs  that  are  embedded  in  a 
weapon  system.  Each  one  of  them  constitutes  an  area  in  which 
complete  studies  could  and  should  be  made. 

Several  essential  areas  of  software  management  have  not 
received  detailed  consideration  in  this  paper.  These  would 
include  cost  control,  configuration  management,  documentation 
standards,  testing  methodology  and  operational  support.  It  is 
recomm.ended  that  future  D3KC  study  projects  address  these 
areas  in  detail. 

A general  recommendation  that  should  be  made  regarding 
software  development  is  akin  to  flag  waving.  Keep  it  simplet 
Computer  programs  should  be  developed  in  an  orderly  and  logical 
manner  in  which  each  task  is  expressed  in  terms  of  simpler 
tasks  until  unique  and  definable  units  are  established  which 
relate  to  higher  level  tasks  through  functional  interface 
definitions.  This  establishes  the  environment  in  which  sound 
management  techniques  can  be  employed.  In  es'^ence  , we  should 
design  software  to  be  --lanageable  and  then  manage  the  design. 
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APPENDIX  A 

INTERVIE'.V  QUESTIONNAIRE 


I 


I? 

i ' 

r 

ft 


I .  Planning 

1.  Guidance  documents? 

2.  Where  in  the  acquisition  cycle  were  s/w  requirenents 
considered?  Documented? 

3.  What  office  manag;es  the  s/w  acquisition? 

4.  WBS  level? 

5.  Cost  Sc  core  estimates? 

6.  3/w  management  visibility?  Konitoring? 

?.  Is  one  rerson  in  ch-^rge? 

8.  Milestones? 

II .  Imrlementation 

1.  When  were  specifications  finalized? 

2.  When  did  coding  start? 

3.  How  was  language  decided  on? 

4.  We'i'e  structured  programming  and/or  PDL  used? 

5.  What  support  s/w  is  required?  How  will  it  be 
documented?  Do  you  have  data  rights? 

6.  Is  s/w  a confi oration  item? 


Ill ,  Eva luati on 

1.  What  is  the  s/w  test  philosophy?  Top-down/£ottom-up? 

2,  What  simulations  are  used?  Are  they  deliverables? 


5.  Will  LBTS  be  used  for  s/w  testiri;^?  How? 

6.  What  s/w  Q.A.  approach  is  used? 

7.  What  is  the  acceptance  criteria? 


IV.  Integration 

1.  Who  has  integr’^tion  r<^sponsibility  for  s/w  and 
hardware? 

2.  What  integration  documents  exist? 

3.  Will  s/w  and  hardware  integration  include  operator/ 
user  integration? 

V,  Life  Cycle  Surrort 

1.  When  was  LC3  planned? 

2.  What  was  the  plan? 

3.  When  did  the  LCS  agent  get  involved? 

4.  'What  considerations  were  given  to  maintainability'? 

5.  Will  the  s/w  be  Service  maintainable? 

VI .  Genera  1 

1.  What  is  the  configuration  control  rlan? 

2.  What  s/w  inputs  were  included  in  D3ARC/GCP? 

3.  'What  is  the  difference  in  s/w  and  hardware  managey^ent 
approaches? 

4.  'What  is  your  position  on  s/w  vs  hardware  flexibility? 

5.  What  nercent  of  s/w  development  cost  went  to  design, 
implementation,  testing,  integration  and  LCS? 

6.  'What  do  you  see  as  the  major  risk  areas  in  s/w 
development? 

j. 
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